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Abstract 


To accommodate byte-range requests for content that has data appended over time, this 
document defines semantics that allow an HTTP client and a server to perform byte-range GET 
and HEAD requests that start at an arbitrary byte offset within the representation and end at an 
indeterminate offset. 


Status of This Memo 


This document is not an Internet Standards Track specification; it is published for examination, 
experimental implementation, and evaluation. 


This document defines an Experimental Protocol for the Internet community. This document is a 
product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF 
community. It has received public review and has been approved for publication by the Internet 
Engineering Steering Group (IESG). Not all documents approved by the IESG are candidates for 
any level of Internet Standard; see Section 2 of RFC 7841. 


Information about the current status of this document, any errata, and how to provide feedback 
on it may be obtained at https://www.rfc-editor.org/info/rfc8673. 


Copyright Notice 


Copyright (c) 2019 IETF Trust and the persons identified as the document authors. All rights 
reserved. 


This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF 
Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this 
document. Please review these documents carefully, as they describe your rights and restrictions 
with respect to this document. Code Components extracted from this document must include 
Simplified BSD License text as described in Section 4.e of the Trust Legal Provisions and are 
provided without warranty as described in the Simplified BSD License. 
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1. Introduction 


Some Hypertext Transfer Protocol (HTTP) clients use byte-range requests (range requests using 
the "bytes" range unit) to transfer select portions of large representations [RFC7233]. In some 
cases, large representations require content to be continuously or periodically appended, such as 
representations consisting of live audio or video sources, blockchain databases, and log files. 
Clients cannot access the appended/live content using a range request with the "bytes" range unit 
using the currently defined byte-range semantics without accepting performance or behavior 
sacrifices that are not acceptable for many applications. 
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For instance, HTTP clients have the ability to access appended content on an indeterminate- 
length resource by transferring the entire representation from the beginning and continuing to 
read the appended content as it's made available. Obviously, this is highly inefficient for cases 
where the representation is large and only the most recently appended content is needed by the 
client. 


Alternatively, clients can access appended content by sending periodic, open-ended byte-range 
requests using the last known end byte position as the range start. Performing low-frequency 
periodic byte-range requests in this fashion (polling) introduces latency since the client will 
necessarily be somewhat behind in transferring the aggregated content, effectively resulting in 
the same kind of latency issues with the segmented content transfer mechanisms in "HTTP Live 
Streaming" (HLS) [RFC8216] and "Dynamic Adaptive Streaming over HTTP" [MPEG-DASH]. While 
performing these range requests at higher frequency can reduce this latency, it also incurs more 
processing overhead and HTTP exchanges as many of the requests will return no content, since 
content is usually aggregated in groups of bytes (e.g., a video frame, audio sample, block, or log 
entry). 


This document describes a usage model for range requests that enables efficient retrieval of 
representations that are appended to over time by using large values and associated semantics 
for communicating range end positions. This model allows representations to be progressively 
delivered by servers as new content is added. It also ensures compatibility with servers and 
intermediaries that don't support this technique. 


1.1. Notational Conventions 


This document cites Augmented Backus-Naur Form (ABNF) productions from [RFC7233], using 
the notation defined in [RFC5234]. 


2. Performing Range Requests on Random-Access Aggregating 
(Live) Content 


This document recommends a two-step process for accessing resources that have indeterminate- 
length representations. 


Two steps are necessary because of limitations with the range request header fields and the 
Content-Range response header fields. A server cannot know from a range request that a client 
wishes to receive a response that does not have a definite end. More critically, the header fields 
do not allow the server to signal that a resource has indeterminate length without also providing 
a fixed portion of the resource. 


A client first learns that the resource has a representation of indeterminate length by requesting 
a range of the resource. The server responds with the range that is available but indicates that 
the length of the representation is unknown using the existing Content-Range syntax. See Section 
2.1 for details and examples. 
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Once the client knows the resource has indeterminate length, it can request a range with a very 
large end position from the resource. The client chooses an explicit end value larger than can be 
transferred in the foreseeable term. A server that supports range requests of indeterminate 
length signals its understanding of the client's indeterminate range request by indicating that the 
range it is providing has a range end that exactly matches the client's requested range end rather 
than a range that is bounded by what is currently available. See Section 2.2 for details. 


2.1. Establishing the Randomly Accessible Byte Range 


Determining if a representation is continuously aggregating ("live") and determining the 
randomly accessible byte range can both be performed using the existing definition for an open- 
ended byte-range request. Specifically, Section 2.1 of [RFC7233] defines a byte-range request of 
the form: 


byte-range-spec = first-byte-pos "-" [ lLast-byte-pos ] 


which allows a client to send a HEAD request with a first-byte-pos and leave last-byte-pos absent. 
A server that receives a satisfiable byte-range request (with first-byte-pos smaller than the 
current representation length) may respond with a 206 status code (Partial Content) with a 
Content-Range header field indicating the currently satisfiable byte range. For example: 


HEAD /resource HTTP/1.1 
Host: example.com 
Range: bytes=0- 


returns a response of the form: 


HTTP/1.1 206 Partial Content 
Content-Range: bytes 0-1234567/x 


from the server indicating that (1) the complete representation length is unknown (via the "*" in 
place of the complete-length field) and (2) only bytes 0-1234567 were accessible at the time the 
request was processed by the server. The client can infer from this response that bytes 0-1234567 
of the representation can be requested and transfer can be performed immediately. 


2.2. Byte-Range Requests beyond the Randomly Accessible Byte Range 


Once a client has determined that a representation has an indeterminate length and established 
the byte range that can be accessed, it may want to perform a request with a start position within 
the randomly accessible content range and an end position at an indefinite/live point -- a point 
where the byte-range GET request is fulfilled on-demand as the content is aggregated. 


For example, for a large video asset, a client may wish to start a content transfer from the video 
"key" frame immediately before the point of aggregation and continue the content transfer 
indefinitely as content is aggregated, in order to support low-latency startup of a live video 
stream. 
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Unlike a byte-range request header field, a byte-content-range response header field cannot be 
"open-ended", per Section 4.2 of [RFC7233]: 


byte-content-range = bytes-unit SP 
( byte-range-resp / unsatisfied-range ) 


byte-range "/" ( complete-length / "x" ) 
first-byte-pos "-" Last-byte-pos 
"x/" complete-Length 


byte-range-resp 
byte-range 
unsatisfied-range 


comp Lete-Length 1*xDIGIT 


Specifically, last-byte-pos is required in byte-range. So, in order to preserve interoperability with 
existing HTTP clients, servers, proxies, and caches, this document proposes a mechanism for a 
client to indicate support for handling an indeterminate-length byte-range response and a 
mechanism for a server to indicate if/when it's providing an indeterminate-length response. 


A client can indicate support for handling indeterminate-length byte-range responses by 
providing a very large value for the last-byte-pos in the byte-range request. For example, a client 
can perform a byte-range GET request of the form: 


GET /resource HTTP/1.1 
Host: example.com 
Range: bytes=1230000-999999999999 


where the last-byte-pos in the request is much larger than the last-byte-pos returned in response 
to an open-ended byte-range HEAD request, as described above, and much larger than the 
expected maximum size of the representation. See Section 6 for range value considerations. 


In response, a server may indicate that it is supplying a continuously aggregating/live response 
by supplying the client request's last-byte-pos in the Content-Range response header field. 


For example: 


GET /resource HTTP/1.1 
Host: example.com 
Range: bytes=1230000-999999999999 


returns 


HTTP/1.1 206 Partial Content 
Content-Range: bytes 1230000-999999999999/x 


from the server to indicate that the response will start at byte 1230000 and continue indefinitely 
to include all aggregated content, as it becomes available. 


A server that doesn't support or supply a continuously aggregating/live response will supply the 
currently satisfiable byte range, as it would with an open-ended byte request. 
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For example: 


GET /resource HTTP/1.1 
Host: example.com 
Range: bytes=1230000-999999999999 


returns 
HTTP/1.1 206 Partial Content 
Content-Range: bytes 1230000-1234567/x 


from the server to indicate that the response will start at byte 1230000, end at byte 1234567, and 
not include any aggregated content. This is the response expected from a typical HTTP server -- 
one that doesn't support byte-range requests on aggregating content. 


A client that doesn't receive a response indicating it is continuously aggregating must use other 
means to access aggregated content (e.g., periodic byte-range polling). 


A server that does return a continuously aggregating/live response should return data using 
chunked transfer coding and not provide a Content-Length header field. A 0-length chunk 
indicates the end of the transfer, per Section 4.1 of [RFC7230]. 


3. Other Applications of Random-Access Aggregating Content 


3.1. Requests Starting at the Aggregation/Live Point 


A client that wishes to only receive newly aggregated portions of a resource (i.e., start at the live 
point) can use a HEAD request to learn what range the server has currently available and initiate 
an indeterminate-length transfer. For example: 


HEAD /resource HTTP/1.1 

Host: example.com 

Range: bytes=0- 
with the Content-Range response header field indicating the range (or ranges) available. For 
example: 


206 Partial Content 
Content-Range: bytes 0-1234567/x 


The client can then issue a request for a range starting at the end value (using a very large value 
for the end of a range) and receive only new content. 


For example: 
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GET /resource HTTP/1.1 

Host: example.com 

Range: bytes=1234567-999999999999 
with a server returning a Content-Range response indicating that an indeterminate-length 
response body will be provided: 


206 Partial Content 
Content-Range: bytes 1234567-999999999999/*x 


3.2. Shift-Buffer Representations 


Some representations lend themselves to front-end content removal in addition to aggregation. 
While still supporting random access, representations of this type have a portion at the 
beginning (the "0" end) of the randomly accessible region that becomes inaccessible over time. 
Examples of this kind of representation would be an audio-video time-shift buffer or a rolling log 
file. 


For example, a range request containing: 


HEAD /resource HTTP/1.1 
Host: example.com 
Range: bytes=0- 


returns 


206 Partial Content 
Content-Range: bytes 1000000-1234567/x 


indicating that the first 1000000 bytes were not accessible at the time the HEAD request was 
processed. Subsequent HEAD requests could return: 


Content-Range: bytes 1000000-1234567/x 


Content-Range: bytes 1010000-1244567/x 


Content-Range: bytes 1020000-1254567/x 
Note though that the difference between the first-byte-pos and last-byte-pos need not be constant. 


The client could then follow up with a GET range request containing: 
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GET /resource HTTP/1.1 
Host: example.com 
Range: bytes=1020000-999999999999 


with the server returning 


206 Partial Content 
Content-Range: bytes 1020000-999999999999/x 


with the response body returning bytes 1020000-1254567 immediately and aggregated/live data 
being returned as the content is aggregated. 


A server that doesn't support or supply a continuously aggregating/live response will supply the 
currently satisfiable byte range, as it would with an open-ended byte request. For example: 


GET /resource HTTP/1.1 
Host: example.com 
Range: bytes=0-999999999999 


returns 


HTTP/1.1 206 Partial Content 
Content-Range: bytes 1020000-1254567/x 


from the server to indicate that the response will start at byte 1020000, end at byte 1254567, and 
not include any aggregated content. This is the response expected from a typical HTTP server -- 
one that doesn't support byte-range requests on aggregating content. 


Note that responses to GET requests performed on shift-buffer representations using Range 
headers can be cached by intermediaries, since the Content-Range response header indicates 
which portion of the representation is being returned in the response body. However, GET 
requests without a Range header cannot be cached since the first byte of the response body can 
vary from request to request. To ensure GET requests without Range headers on shift-buffer 
representations are not cached, servers hosting a shift-buffer representation should either not 
return a 200-level response (e.g., send a 300-level redirect response with a URI that represents the 
current start of the shift buffer) or indicate the response is non-cacheable. See [RFC7234] for 
details on HTTP cache control. 


4. Recommendations for Byte-Range Request last-byte-pos 
Values 


While it would be ideal to define a single large last-byte-pos value for byte-range requests, there's 
no single value that would work for all applications and platforms. For example, JavaScript 
numbers cannot represent all integer values above 2453, so a JavaScript application may want 
to use 24453-1 for last-byte-pos. This value, however, would not be sufficient for all applications, 
such as long-duration high-bitrate streams. So 24453-1 (9007199254740991) is recommended as a 
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last-byte-pos unless an application has a good justification to use a smaller or larger value. For 
example, if it is always known that the resource won't exceed a value smaller than the 
recommended last-byte-pos for an application, a smaller value can be used. If it's likely that an 
application will utilize resources larger than the recommended last-byte-pos (such as a 
continuously aggregating high-bitrate media stream), a larger value should be used. 


Note that, in accordance with the semantics defined above, servers that support random-access 
live content will need to return the last-byte-pos provided in the byte-range request in some 
cases -- even if the last-byte-pos cannot be represented as a numerical value internally by the 
server. As is the case with any continuously aggregating/live resource, the server should 
terminate the content transfer when the end of the resource is reached -- whether the end is due 
to termination of the content source or the content length exceeds the server's maximum 
representation length. 


5. IANA Considerations 


This document has no IANA actions. 


6. Security Considerations 


As described above, servers need to be prepared to receive last-byte-pos values in range requests 
that are numerically larger than the server implementation supports and return these values in 
Content-Range response header fields. Servers should check the last-byte-pos value before 
converting and storing them into numeric form to ensure the value doesn't cause an overflow or 
index incorrect data. The simplest way to satisfy the live-range semantics defined in this 
document without potential overflow issues is to store the last-byte-pos as a string value and 
return it in the byte-range Content-Range response header's last-byte-pos field. 
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